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In this article we investigate the role of connectivity in promoting coherent activity in excitatory 
neural networks. In particular, we would like to understand if the onset of collective oscillations 
can be related to a minimal average connectivity and how this critical connectivity depends on the 
number of neurons in the networks. For these purpouses, we consider an excitatory random network 
of leaky integrate-and-fire pulse coupled neurons. The neurons are connected as in a directed Erdos- 
Renyi graph with average connectivity (fc) scaling as a power law with the number of neurons in 
the network. The scaling is controlled by a parameter 7, which allows to pass from massively 
connected to sparse networks and therefore to modify the topology of the system. At a macroscopic 
level we observe two distinct dynamical phases: an Asynchronous State (AS) corresponding to a 
desynchronized dynamics of the neurons and a Partial Synchronization (PS) regime associated with 
a coherent periodic activity of the network. At low connectivity the system is in an Asynchronous 
State, while PS emerges above a certain critical average connectivity (fc)c. For sufficiently large 
networks, {k)c saturates to a constant value suggesting that a minimal average connectivity is 
sufficient to observe coherent activity in systems of any size irrespectively of the kind of considered 
network: sparse or massively connected. However, this value depends on the nature of the synapses: 
reliable or unreliable. For unreliable synapses the critical value required to observe the onset of 
macroscopic behaviors is noticeably smaller than for reliable synaptic transmission. Due to the 
disorder present in the system, for finite number of neurons we have inhomogeneities in the neuronal 
behaviors, inducing a weak form of chaos, which vanishes in the thermodynamic limit. In such a 
limit the disordered systems exhibit regular (non chaotic) dynamics and their properties correspond 
to that of a homogeneous fully connected network for any 7-value. Apart for the peculiar exception 
of sparse networks, which remain intrinsically inhomogeneous at any system size. 

PACS numbers: 05.45.Xt, 84.35.+i, 87.19.lj, 87.19. In 



The spontaneous emergence of collective dy- 
namical behaviours in random networks made of 
many (identical) interacting units is a subject of 
interest in many different research fields rang- 
ing from biological oscillators to power grids. In 
particular, how the macroscopic dynamics of the 
network is influenced by the topology is an ac- 
tive research line not only for nonlinear dynam- 
ics, but also for many other scientific disciplines, 
as for example (computational) neuroscience 
However, the most part of the performed analy- 
sis have been devoted to the emergence of the 
fully synchronized regimes, but in neuroscience a 
complete synchronization is usually a symptom 
of neural disorders, while coherent oscillations 
are often associated to a partial synchronization 
among neurons during brain activity [8]. Coher- 
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ent oscillatory activities are prominent in the cor- 
tex of the awake brain during attention, and have 
been implicated in higher level processes, such as 
sensory binding, storage of memories, and even 
consciousness . 



In this article, we analyse how the presence 
of disorder in the connections can influence the 
emergence of coherent periodic activity in Erdos- 
Renyi networks of excitatory pulse coupled spik- 
ing neurons. Our main result indicates that the 
parameter controlling the transition from asyn- 
chronous to coherent neural activity is simply 
the average connectivity. Furthermore, for (suffi- 
ciently large) networks the critical value of the av- 
erage connectivity turns out to be independent of 
the network realization, sparse or massively con- 
nected, but it is instead influenced by the nature 
of the disorder, quenched or annealed. 
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I. INTRODUCTION 

Neural collective oscillations have been observed in 
very many context in brain circuits, ranging from ubiqui- 
tous 7 oscillations to 9 rhythm in the hippocampus. The 
origin of these oscillations is commonly associated with 
the balance between excitation and inhibition in the net- 
work, while purely excitatory circuits are believed to lead 
to " unstructured population bursts" Q. However, recent 
'ex vivo" measurements performed on the rodent neocor- 
tex and hippocampus ^] in the early stage of brain 
maturation reveal coherent activity patterns, such as Gi- 
ant Depolarizing Potentials. These collective oscillations 
emerge despite the fact that the GABA transmitter has 
essentially an excitatory effect on immature neurons 
Therefore, also in purely excitatory networks one can ex- 
pect non trivial dynamics at a macroscopic level. 

Numerical and theoretical studies of collective mo- 
tions in networks of simple spiking neurons have been 
mainly devoted to balanced excitatory-inhibitory config- 
urations, e.g see Q and references therein. Only few 
studies focused on coherent periodic activity in fully cou- 
pled excitatory networks of leaky integrate-and-fire (LIF) 
neurons. These analysis revealed a regime character- 
ized by a partial synchronization (PS) at the population 
level, while the single neurons perform quasi-periodic mo- 
tions [l^. It has been shown that the PS regime is quite 
robust to perturbations, since it survives to moderate 
levels of noise or dilution [13, [i3| • 

Furthermore, in their recent study Bonifazi et al. 
found that the functional connectivity of developing 
hippocampal networks is characterized by a truncated 
power-law distribution of the out-degrees with exponent 
7 = 1.1 — 1.3. This scaling has been shown to hold over 
one/two decades, thus not ensuring a scale- free distribu- 
tion for the links over all scales, but surely indicating 
the presence of a large number of hub neurons, namely 
cells characterized by a high connectivity. At early devel- 
opmental stages of the brain, GABAergic hub interneu- 
rons, performing complex excitatory/shunting inibitory 
actions Q, seem to be responsible for the orchestration 
of the coherent activity of hippocampal networks @ . The 
relevance of hubs in rendering a neural circuit extremely 
hyperexcitable has been also demonstrated in simulation 
studies of a realistic model of the epileptic rat dentate 
gyrus, even in the absence of a scale- free topology \X^. 

Motivated by these studies, but without attempting to 
reproduce the experimental results, we focus on a very 
preliminary question: to which extent is the macroscopic 
neural dynamics influenced by the average degree of con- 
nectivity of the neurons ? Our specific aim is to analyze 
the key ingredients leading to the onset of coherent activ- 
ity, as opposed to asynchronous dynamics, for different 
network size and topology. 

Specifically, we consider the transition from an asyn- 
chronous regime to partial synchronization in the exci- 
tatory LIF pulse coupled neural networks introduced by 
Abbott and van Vreeswijk [l|. At variance with previ- 



ous works, we consider dynamical evolution on random 
Erdos-Renyi (ER) networks with an average connectivity 
corresponding to that of a truncated power-law distribu- 
tion with a decay exponent 1 < 7 < 2. This amounts to 
have an average connectivity which scales proportionally 
to N'^ '' with the number of neurons in the network. In 
the limit 7 — >■ 1 the massively connected network, where 
the connectivity is proportional to iV, is recovered; while 
for 7 — > 2 a sparse network, where the average probabil- 
ity to have a link between two neurons vanishes in the 
thermodynamic limit [To| . is retrieved. The topology of 
ER networks is modified by varying the parameter 7 in 
the interval [1 : 2], in particular as far as 7 < 2 trees 
and cycles of any order are present in the network, while 
for 7 — 1 complete subgraphs of increasing order ap- 
pear in the system j2|. In the paper we will study how 
and to which extent these topological modifications in- 
fiuence the macroscopic dynamics of the network, with 
particular emphasis on the transition from asynchronous 
to (partially) synchronous collective dynamics. 

The paper is organized as follows: the next Section is 
devoted to the introduction of the neural model and of 
the indicators employed to characterize the dynamics of 
the network. The phase diagram reporting the collective 
states emerging in our system is described in Sect. 3. 
The influence of finite size effects on the coherent activity 
is analyzed in Sect. 4, while a characterization of the 
neural dynamics in term of maximal Lyapunov exponent 
is reported in Sect. 5. A brief discussion of our results is 
outlined in Sect. 6. 



II. MODEL AND METHODS 

A. The Model 

We study a network of N LIF neurons with the mem- 
brane potential Xi{t) G [0 : 1] of the neuron i evolving 
as: 

x,{t) ^ a - x,{t) + h{t) i = l,---,N , (1) 

where a > 1 is the suprathreshold DC current, li the 
synaptic current. Whenever the neuron reaches the 
threshold = 1, a pulse s{t) is instantaneously trans- 
mitted to all the connected post-synaptic neurons and 
the membrane potential of neuron i is reset to Xi — 0. 
The synaptic current can be written as Ii{t) = gEi{t), 
with g > representing the synaptic excitatory strength 
while the field Ei{t) is the linear superposition of the 
pulses s{t) received by neuron i in the past, in formula 

= C,^^e{t~tn)s{t~tn) , (2) 

' n|t„<t 

where ki is the number of pre-synaptic neurons connected 
to the neuron i (i.e. the in-degree of neuron i) and 0{t) 
the causal function. The connectivity matrix Cj^i ap- 
pearing in Eq. ^ has entry 1 (resp. 0) depending if the 
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pre-synaptic neuron j is connected (resp. not connected) 
to neuron i and in general it is not symmetric. 

Following van Vreeswijk [23| we assume, for the single 
pulse emitted at t = 0, the shape s{t) — a^t exp{—at). 
The explicit equation ([2]) can be thus rewritten as an 
implicit ordinary differential equation: 



for the truncated (power-law) distribution is: 



E,{t) + 2aE,{t) + a^E.it) = p ^ CjJ{t - i„) . (3) 

n|t„(t 



The continuous time evolution of the network can be 
transformed in a discrete time event-driven map by inte- 
grating Eq. ([3]) from time tn to time in+i, i„ being the 
time immediately after the n-th spike emission. 

Following Olmi et al. [T7| . the event-driven map read 
as: 

E,{n + l) = S,(n)e-"^(") +g,(n)T(n)e-""(") (4) 



(5) 



x,(n + l) = a;,(n)e-^(") +a(l - e-^(")) +.giJ,(n)(6) 

where Qi = aEi -\- Ei is an auxilary variable. Hi — 
Hi{Ei,Qi,T) is a nonlinear function and r(n) = tn+i — tn 
is the interspike time interval. This can be determined 
by solving the following implicit relationship 



T{n) = In 



a -f gHm{n) - 1 



(7) 



where m(n) identifies the neuron which will fire next at 
time tn+i by reaching the threshold value Xm = 1- 

The evolution of the system is now modeled with a 
discrete time map of 3A^ — 1 variables, {Ei,Qi,Xi}. In 
fact the degree of freedom associated with the membrane 
potential of the firing neuron has been removed by the 
implementation of the Poincare-section, leading to the 
event driven map. More details on the model are reported 
in Ref. [l7j; however at variance with that study the 
pulse amplitudes, appearing in eqs. (HI,© and (O, are 
normalized by the in-degree fc.; of neuron i and not by 
the total number of neurons N . 

The model parameters were fixed as a = 1.3, 5 = 0.4 
and a = 9, in order to ensure the emergence of a PS 
regime in the corresponding fully coupled network [23| • 



B. The Connectivity Matrix 

Our choice for the connectivity matrix has been mainly 
motivated by the results obtained for "ex vivo" cells by 
Bonifazi et al. In particular, these authors have 

shown that in developing hippocampal networks the func- 
tional connectivity is distributed according to power-law 
distribution, at least over one/two decades. The distribu- 
tion of links k per neuron is thus given by P{k) = pk~'^ , 
where p is a normalization constant. Therefore, for a fi- 
nite network made of N neurons the average connectivity 



(fc> 



7 



[N^-'' - 1] 



(8) 



Furthermore, Bonifazi et al @ have measured quite low 
values for the exponent 7, namely 7 = 1.1 — 1.3, sug- 
gesting the existence of a large number of highly con- 
nected neurons, hubs, in the network. The role of hub 
neurons in orchestrating the level of synchrony in living 
brain networks has been clearly demonstrated in [()]. In 
spite of these findings the underlying network topology is 
maybe less important than the actual number of highly 
connected neurons, as suggested by Morgan and Soltesz 

In order to single out the effect of the average num- 
ber of connections on the network dynamics we decided 
to limit our analysis to Erdos-Renyi (ER) random net- 
work but with an average connectivity given by Eq. 
([HI). In particular, we considered a directed ER random 
graph, where the distribution of links is well approxi- 
mated by a Poisson distribution 3j namely: 



P(fc) 



-{k) 



ki 



(9) 



According to Eq. © the degree distribution is com- 
pletely defined once the value of (fc) is given. As a matter 
of fact, by choosing for (fc) the expression ([5]), the prob- 
ability of existence of an unidirectional link connecting 
neuron j to i (i.e. the probability to have Cj^i = 1) is: 



Pr(7V,7) 



(fc) 
N 



2-7 



1 

N 



(10) 



In the limit 7 — >■ 1 the massively connected network 
is recovered [l3|, since the average connectivity (fc) — 
p X (N — 1) is proportional to the system size and 
Pr{N,l) ~ p{l — 1/N) is, apart finite-size corrections, 
constant and coincident with p. For 7 < 2 (resp. 7 > 2) 
the average number of synaptic inputs per neuron will 
grow (resp. decrease) with N, in the limiting case 7 = 2 
a sparse network will be essentially recovered [22] since 
(fc) = phiN will vary in a limited manner with respect 
to the system size. Indeed, by varying N by three or- 
ders of magnitude from 100 to 100,000 the value of (fc) 
will modify from 3.7 to 9.2 with p = 0.8. In the follow- 
ing, we studied networks of various sizes N, ranging from 
TV = 100 to = 200, 000, for different 7- values in the in- 
terval [1:2]. The value p is usually fixed to 0.8, apart for 
7 = 1 (constant probability case) where the dependence 
on p is examined in details in Section 3. 

Let us stress that in the present study the distributions 
of pre-synaptic (in-degree) and of post-synaptic (out- 
degree) connections are identical, and this is guaranteed 
by the above outlined procedure to determine unidirec- 
tional links for ER networks. 

In what follows, we consider two different ways to se- 
lect the random connectivity matrix: in the first case 
the synaptic connections are randomly chosen and are 
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constant in time (quenched disorder); while for the sec- 
ond procedure the neurons receiving the excitatory pulse 
are randomly selected each time a neuron fires {annealed 
disorder). The latter choice can be justified from a physi- 
ological point of view by the fact that the synaptic trans- 
mission of signals is an unreliable process It should 
be noticed that in the annealed case, since the network 
modifies in time, the pulse amplitudes are normalized by 
the average in-degree (fc) and not by ki as in the quenched 
case. 



C. Characterization of Macroscopic Attractors 

In order to perform a macroscopic characterization of 
the dynamical states of the network wc exploit the aver- 
age fields: 



^w = fE^'W ' qw = ^EQ'W' (11) 

i=l 1=1 

where the fields have been also rescaled by the synaptic 
strength as done in Ref. flT\ . 

As a measure of the level of homogeneity among the 
neurons of the network, we consider the standard devia- 
tion a{t) of the fields {Ei{t)} acting on the single neurons 



N 



1/2 



(12) 



for completely homogeneous systems, such as globally 
coupled networks, Ei{t) = E and cr = 0. 

The degree of synchronization among the neurons is 
quantified by the order parameter usually employed in 
the context of phase oscillators J^] 



R{t) 



1 ^ 
N ^ 



(13) 



where 9j is the phase of the j neuron, that can be 
properly defined as a (suitably scaled) time variable [2l| , 
9j{t) = 2n(t — tj^ri)/Tq^rn whcrc „ indicates the time 
of the last spike emitted by the j**^ neuron, while Tg.„ = 
tq,n+i — tq,n IS the 7T,-th iutcrspike interval associated to 
the neuron q which was the last to fire in the network 

m. 

A non zero- value of R represents an indication of par- 
tial synchronization among the neurons, while a vanish- 
ingly small R ^ 1/a//V is observable for asynchronous 
states in finite systems 



D. Lyapunov Analysis 

The dynamical microscopic instabilities of a system 
can be characterized in terms of the maximal Lyapunov 



exponent A: a positive A being a measure of the degree 
of chaoticity of the considered system. In particular, we 
have employed the standard method developed in Ref. 
[l9( to estimate the maximal Lyapunov exponent by fol- 
lowing the evolution of an infinitesimal perturbation to 
a reference trajectory. 



III. PHASE DIAGRAM 

In this model two different macroscopic regimes can be 
observed: the asynchronous state (AS) and the partial 
synchronization (PS). AS is characterized by an incoher- 
ent dynamics of the neurons in the network leading to a 
spot-like macroscopic attractor in the [E, Q) plane and 
an almost constant average field E [l| , while to the coher- 
ent PS regime corresponds a closed curve attractor [l3| 
and a periodic behavior of E in time as shown in Fig. 
[1^ and b. The incoherent and coherent neural dynamics 
can be clearly appreciated, in the two regimes, also at 
microscopic level by examining the corresponding raster 
plots reported in Fig. [T}; and[T}i. 

As a first aspect, we will investigate the occurrence 
of AS and PS for finite networks and different average 
connectivity (k) . In order to distinguish the two regimes 
we have examined the macroscopic attractor shape, the 
extrema values of the average field E, and the synchro- 
nization parameter i? as a function of N. 



A. ER networks with constant probability 

As an initial reference study we consider ER networks 
with unidirectional links chosen at random with a con- 
stant probability (CP) p for any network size N. This 
amounts to consider the limiting case 7—^1 and to have 
an average connectivity scaling linearly with the size, i.e. 
(fc) = pN. Let us firstly examine the minima and max- 
ima of i5 of a network of size N by varying the probability 
p between and 1, as shown in Fig. ^j^ioi N — 1, 600. At 
low probability one has AS while the PS regime emerges 
only for sufficiently large p, both in the quenched and 
annealed case. This result is related to the fact that the 
presence of noise reduces the coherence needed to observe 
the PS regime [lj|. As a matter of fact increasing p (i.e. 



diminishing the number of broken links in the network) 
the attractor size increases and finally reaches the fully 
coupled result. The degree of coherence can be measured 
in terms of the average synchronization indicator (i?) . As 
shown in Fig. [^b the system coherence steadily increases 
with p, except in the AS regime where (i?) ^ 0, apart for 
finite size fluctuations. 

Let us now report the phase diagram for the macro- 
scopic activity of the network in the {N, (k)) plane, for 
both annealed and quenched disorder. Increasing the av- 
erage connectivity, keeping the system size fixed, leads 
to a transition from AS to PS regimes (see Fig. The 
transition occurs at a critical average connectivity (fc)c. 
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FIG. 1: (Color online) AS and PS characterization in terms 
of macroscopic fields and single neuron dynamics. Panel a: 
macroscopic attractors in the {E, Q) plane. Panel b: the av- 
erage field i? as a function of time. Panel c and d: raster 
plots. The data refer to ER networks with (k) — p x N, 
quenched disorder, and A'^ = 1, 600, the black (resp. red) 
symbols correspond to AS observable for p — 0.2 (resp. PS 
for p = 0.7). 



indicates by the asteriskes connected by a black solid line 
in Fig. [31 which for low N increases steadily with N, but 
eventually saturates for N > 10, 000 to an asymptoti- 
cally constant value which depends on the noise realiza- 
tion: namely, (fc)as = 725 ±25 for the quenched case and 
{k)as = 225 ± 25 in the annealed one |24| . 



B. ER networks with 7-dependent probability 

To verify the generality of these results we investigate 
ER networks with 7-dependent probability. In partic- 
ular, we have estimated, for system sizes in the range 
100 < N < 200, 000, the macroscopic attractors for var- 
ious 7- values (namely, 7 = 1.1, 1.3, 1.5 and 1.7), after 
discarding long transient periods. For small system sizes 
the network is in the AS regime which is characterized by 
a spot-like attractor in the {E, (5)-plane. For larger num- 
ber of neurons, PS emerges in the system characterized 
by closed curve (macroscopic) attractors. Furthermore, 
similarly to the results reported in Ref. , increasing N 
the curves tend to an asymptotic shape, corresponding to 
the fully coupled attractor, while fluctuations diminish. 
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FIG. 2: (Color online) a) Minima and maxima of the av- 
erage field _E as a function of p for N — 1,600, the circles 
joined by solid lines refer to the annealed disorder, while the 
stars connected by the dashed line to the quenched case. The 
dot-dashed (red) lines indicate the fully coupled results (cor- 
responding to p = 1). b) Synchronization indicator (R) aver- 
aged over time as a function of p for the quenched case with 
A'' = 3, 200. The data refer to ER networks with constant 
probability (CP) and have been estimated, after discarding 
a transient of 4 x 10^ spikes, by averaging over a train of 
1 - 2 X 10^ spikes. 



To exemplify this point various macroscopic attractors 
for 7 = 1.3 and annealed disorder are reported in Fig. 

Hi. 

As already reported for the CP networks the systems 
with annealed disorder converge more rapidly with N to- 
wards the asymptotic fully coupled attractor with respect 
to the quenched shown in Fig. |4)d for 7 = 1.1, 

1.3, 1.5. Increasing 7 we observe that the transition from 
AS to PS occurs at larger and larger system size, both for 
annealed and quenched disorder. The results for 7 = 1.7 
are not shown in Fig. |4l3, since for all the examined 
network sizes (up to A^ = 200, 000) E extrema coincide 
within the error bar, indicating that the system is in the 
AS regime ^ . 

AS (resp. PS) regimes are reported in the phase 
diagram displayed in Fig. [3] as filled (resp. empty) 
symbols for the investigated 7-values. The critical line 
{k)c = {k)c{N) (indicated by the solid black line in Fig. 
[3]) denoting the transition from AS to PS coincides with 
that determined previously for CP networks. We can 
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FIG. 3: (Color online) Phase diagram for the macroscopic 
activity of the network in the (A'', (k)) plane: a) annealed dis- 
order and b) quenched disorder. The (black) asteriskes con- 
nected by the solid (black) line correspond to the transition 
values {k)c from AS to PS regime estimated for ER networks 
with CP. The other symbols refer to ER with 7 > 1: solid 
(resp. empty) symbols individuate AS (resp. PS) states. In 
particular, (blue) squares refer to 7 = 1.1, (red) circles to 
7 = 1.3, (green) diamonds to 7 = 1.5, and (magenta) trian- 
gles to 7 = 1.7. The reported data are relative to the state of 
the network after discarding transients ranging from 2 x 10^ 
spikes at the smaller sizes to 3 x 10* spikes for the larger 
networks. 



thus safely afRrm that the dynamical regimes of the ER 
networks depend, at a macroscopic level, simply on the 
average connectivity, once the system size N is fixed. 
This could be expected from the fact that for ER net- 
works the distribution of links per neuron is completely 
determined by (k) (see Eq. (|9])). However, the indepen- 
dence of {k)c from N at N > 10,000 is unexpected and 
suggests that coherent behaviors, like PS regimes, can 
be observed in networks of any size for vanishingly small 
relative connectivity (fc) /N. 



IV. NETWORK HOMOGENEITY 

We observe that for any considered 7 the fields {Ei, Qi) 
associated with the different neurons tend to synchronize 
for increasing N . Therefore in the thermodynamic limit 
{N ^ 00) disordered networks tend to behave as fully 
coupled ones, where all the neurons are equivalent and a 
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FIG. 4: (Color online) a) Macroscopic attractors in the [E, Q) 
plane for a ER networks with 7 = 1.3 and annealed disorder, 
the curves from the interior to the exterior corresponds to 
increasing system sizes, from A'' = 1, 600 to 25,600. The most 
external (black) curve refers to a fully coupled network with 
= 3, 200. b) Minima and maxima values of the average field 
£ as a function of for various 7-values: namely, (black) 
circles 7 = 1.1, (red) triangles 7 = 1.3 and (blue) squares 
7 = 1.5. The empty (resp. filled) symbols refer to annealed 
(resp. quenched) disorder. The dot-dashed (magenta) lines 
indicate the fully coupled values. 



single field is sufficient to describe the macroscopic evo- 
lution of the system. 

In order to quantify the level of homogeneity among 
the various neurons, we measured the standard devia- 
tion (Eq. ([H])) relative to the fluctuations of the differ- 
ent local fields Ei with respect to the average field E. In 
Fig. [S^ we plot the time average of the standard devia- 
tion, (ct), for annealed disorder and various 7 values. We 
observe a power law decay (cr) cx N^^ , where the expo- 
nent /3 depends on the 7-parameter as/3 = l — 7/2 (see 
Fig. [S})). Furthermore, for the liming case 7 = 2.0 the 
decay of (a) is consistent with a scaling l/vmJV as dis- 
played in the inset of Fig. [S^. Alltogether, the reported 
dependencies suggest the following relationship to hold: 



(cr) CX 



1 



(14) 



thus fields fluctuations are driven by the average in- 
degree value irrespectively of the total number of neu- 
rons. These results confirm once more that in the limit 
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FIG. 5: (Color online) a) Average standard deviation (a) 
versus the system size A'^ for annealed disorder and various 
7-values: 7 — 1.1 (blue) squares, 1.3 (red) circles, 1.5 (green) 
diamonds and 1.7 (magenta) triangles. The dashed line rep- 
resents best fits with a power-law N~'^ to the reported data. 
The data in the inset (black asterisks) refers to a 7 = 2.0, the 
dashed line is a guide for the eyes, b) Power-law exponents /3 
for annealed (black circles) and quenched (red triangles) dis- 
order in the network as a function of the parameter 7. The 
dashed (blue) line refers to the linear law /3 = 1 — 7/2. The 
reported data have been estimated by averaging over trains 
made of 2 x lO'' — 10* spikes, after discarding transients of 
4 X 10^ - 4 X 10^ spikes. 



N ^ 00 the neural field dynamics converges to that of 
homogeneous networks, for both quenched and annealed 
disorder. 

The relationship among (a) and the average connectiv- 
ity reported in eq. clearly indicates that for a sparse 
network, with constant average connectivity, (cr) will re- 
main finite even in the thermodynamic limit. Therefore, 
we can conclude that a sparse network cannot be ever 
reduced to a fully coupled one by simply rescaling the 
synaptic coupling as done in Ref. [17|, even for very 
large systems. 



V. CHAOTIC VS REGULAR DYNAMICS 

Homogeneous fully connected pulse coupled networks 
exhibit regular dynamics (20| . In particular, for excita- 
tory network and finite pulses the AS becomes a splay 
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FIG. 6: (Color online) Maximal Lyapunov exponents as a 
function of the system syze A'' for various 7-values. The data 
have been obtained by discarding a transient of the order of 
10* — 10^ spikes and then by following the dynamics in the 
real and tangent space for an equivalent duration, moreover 
the data have been averaged over 3 to 5 different network 
realization with quenched disorder. 



state characterized by all neurons spiking one after the 
other at regular intervals with the same frequency and by 
a constant mean field E, while the PS regime becomes 
perfectly periodic at a macroscopic level [l| . 

The introduction of disorder in the network leads to 
irregularity in the dynamics of the single neurons, which 
are reflected also at a macroscopic level. This kind of de- 
terministic irregular behavior has been identified as weak 
chaos whenever the irregularity vanishes for sufficiently 
large system size [l^. The chaotic motion can be char- 
acterized in terms of the maximal Lyapunov exponent A: 
regular orbits have non positive exponents, while chaotic 
dynamics are associated with A > 0. 

For finite size networks chaotic dynamics is observed 
both for annealed and quenched disorder. However, for 
all 7-values examined in this work, A tends to decrease 
for a sufficiently large number of neurons in the network. 
Therefore we can safely affirm that for any ER network 
with average connectivity given by Eq. ([8]) the neuronal 
dynamics is weakly chaotic; i.e. the evolution will be- 
come completely regular for infinite networks. Numeri- 
cal results for quenched disorder are reported in Fig. [6^ 
for various 7-values. The maximal Lyapunov exponent 
exhibits clear power-law decays N"^ , with S decreasing 
and eventually vanishing for 7 — >■ 2 (see Table |T| . These 
results generalize previous indications reported in Olmi 
et al. for a specific realization of diluted network [l7| . 

Considering networks with annealed disorder it must 
be underlined that for sufficiently large number of neu- 
rons and 7 < 2, the maximal Lyapunov reveals a ten- 
dency to decrease. However, clear scaling laws cannot 
be inferred for all the examined 7-values on the range 
of affordable system sizes, namely 50 < < 12, 500. 
Larger network sizes are probably required with unreli- 
able synapses to have clear scaling laws, due to the fact 
that the transition from AS to PS occurs at critical sys- 
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TABLE I: Power law exponents giving the decay of the max- 
imal Lyapunov A oc 7V~* for the data reported in Fig. [S^ for 
quenched disorder. 

7 S 

1.1 0.26 ±0.01 

1.3 0.228 ± 0.007 

1.5 0.174 ±0.006 

1.7 0.085 ± 0.009 



tern sizes within the investigated range. On the contrary, 
in the quenched case once 7 is fixed an unique regime is 
observable for almost all the investigated sizes. In par- 
ticular, the network is always in an AS for 7 = 1.7 and 
1.5, while it is in the PS regime for 7 = 1.1 and 1.3 for 
(almost) all the considered number of neurons. These 
additional findings strengthen the above reported con- 
clusions: finite size systems are weakly chaotic for any 
considered dynamical regime. 

According to the results reported above, we expect 
that for sparse networks the maximal Lyapunov exponent 
will eventually saturate to some constant value, apart for 
possible logarithmic corrections. Thus, sparse networks 
should remain chaotic for any system size, paralleling 
the behavior of the microscopic fluctuations shown in the 
previous Section. Therefore microscopic inhomogeneities 
and chaotic behavior appear as deeply related. 

VI. DISCUSSION 

Collective periodic oscillations in excitatory ER net- 
works can be observed only above a critical average in- 
degree. This latter quantity saturates to a constant value 
for networks with a sufficiently large number of neurons, 
thus suggesting that the key ingredient responsible for 
the emergence of collective behaviors is the number of 
pre-synaptic neurons both for massively connected net- 
works as well as for sparsely connected ones. This result 
confirms and generalizes previous findings on the stabil- 
ity of the complete synchronized state for pulse-coupled 
Hindmarsh-Rose neurons Furthermore, our results 
indicate that the minimal network size required to ob- 
serve collective oscillations diverges with the exponent 7 
ruling the scaling of the average connectivity with the 
number of neurons. 

The presence of annealed disorder in the network (cor- 
responding to unreliable synapses) favors the emergence 
of coherent activity with respect to the quenched case 
(associated with reliable synapses), since in the first case 
the asymptotic average connectivity required to observe 
collective oscillations is much smaller. This is probably 
due to the fact that in the annealed situation each neuron 
is on average subjected to the same train of stimuli, while 
with quenched disorder the dynamics of each neuron de- 



pends heavily on its neighbors. Furthermore, for suffi- 
ciently large networks the macroscopic behavior observ- 
able with reliable or unreliable synapses becomes identi- 
cal. This seems to indicate that at the level of population 
dynamics the reliability or unreliability in the synaptic 
transmission can be irrelevant. 

The average in-degree (k) also controls the fluctuations 
among different neurons, being the fluctuations propor- 
tional to the inverse of the square root of (fc). There- 
fore, for ER networks with average in-degree proportional 
to any positive power of N, the fluctuations will vanish 
in the limit iV — > 00, leading to a homogeneous collec- 
tive behavior analogous to that of fully connected net- 
works. On the contrary, inhomogeneities among neurons 
will persist at any system size in sparse networks. 

Recent experimental results on the intact neuronal net- 
work of the barrel cortex of anesthetized rats seems to 
clearly suggest that the dynamics of this system is chaotic 
[Tst . This result poses severe questions about the possi- 
bility of reliable neural coding. The evolution of our mod- 
els is chaotic for any finite networks. However, in pres- 
ence of coherent periodic activity, the chaoticity present 
in the system is not so strong to destroy the average col- 
lective motion. Thus the trial-to-trial variability induced 
by chaos does not prevent the possibility of the network 
to encode information. The information can be coded 
in some property associated to the global oscillations of 
the network which are robust to local fluctuations, and 
this collective coding can represent an alternative to the 
dilemma rate versus spike timing coding 13 ] . 

The level of chaos in the examined networks decreases 
with N and the dynamics becomes regular in the ther- 
modynamic limit. However, the decrease of the maximal 
Lyapunov exponent with N slows down dramatically for 
7—^2, suggesting an erratic asymptotic behavior for 
sparse networks. 

Our results represent only a first step in the analysis 
of what matters in the network topology for the emer- 
gence of coherent neural dynamics. In future works, our 
findings should be critically verified for more complex 
topologies, like scale- free and small- world, and the infiu- 
ence of other ingredients, like the asymmetry in the in- 
degree and out-degree distributions [IS], should be also 
addressed. 
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A sparse network is usually defined as a network where 
the average connectivity remains constant in the thermo- 
dynamic limit, see [1jO| |. 

In the original definition reported in [2l[, Tq^n = Tj^n, 
thus ensuring that the phase is bounded between and 
2tt. However this definition requires to memorize all the 
spiking events within a certain time interval and then 
estimate the phases and R{t) over such time span. Our 
definition does not guaratee that the phase is strictly 
bounded within the [0 : 27r]-interval, but it allows to esti- 
mate R{t) during the simulation run without memorizing 
any event. Indeed we have verified that, at least in the 
studied cases, no peculiar differences are observable by 
employing the two phase definitions and this is probably 
due to the fact that the dynamics of all the neurons in 
the networks are statistically equivalent. 
We have verified that the above reported scenario is not 
modified by considering fully homogeneous connectivity 
with P{k) = S{k — fco), the only difference concerns the 
asymptotic value {k}a3 at which the transition from AS 
to PS occurs. As a matter of fact {k)as is slightly smaller 
than for ER distributions due to the absence of fluctu- 
ations among the in-degree values in the homogeneous 
case. 

A simple estimation of the minimal network size needed 
to observe PS for generic 7-distributions can be obtained 
by inverting Eq. ((8]) for the asymptotic values {k}as, 
namely 



■7 



-{k)aB + 1 



1 1/(2-7) 



(15) 



for 7 = 1.7 this leads to iV^ = 131,784,000 (resp. 
Nc — 2, 740, 117) in the quenched (resp. annealed) case. 
This confirms that coherent activity is unobservable for 
7 = 1.7 with our computational resources for the chosen 
coupling parameter, namely g = 0.4. 



